perf(qwen4): optimize gated residual kernels and PDL - #1323

Summary

• Move the Qwen4 gated-residual chain behind GPU-only tokenspeed-kernel APIs. Runtime code now owns parameters and checkpoint mapping only; grouped RMSNorm, mix, and combine implementations live in tokenspeed-kernel.
• Fuse the down and inject projections, add a low-M persistent Triton path plus a portable Triton path, and keep the Blackwell CuTeDSL composition available for explicit tuning. Dispatch falls back to the deterministic general path when atomic accumulation is disabled.
• Use stream-private persistent workspaces, preserve reduce-scatter row views, and apply non-power-of-two projection scaling after the projection to avoid checkpoint-weight quantization.
• Prepare CuTeDSL's rank-padded derived weights once outside forward/CUDA Graph capture. Their shape and address stay fixed; supported online weight synchronization refreshes the same allocation through the weight loader, while forward performs a read-only lookup and rejects unprepared state. Validate grouped RMSNorm output buffers consistently for empty and non-empty inputs.
• Remove redundant runtime CPU/CUDA branches, the registered CPU reference backend, and the public mix-epilogue wrapper. Numerical references remain test-only.

Performance

Compared with baseline 63f4585 on an NVIDIA B200 using BF16, PDL enabled, and CUDA Graph replay. The benchmark covers the full runtime grouped RMSNorm + gated-residual mix + combine chain; values are medians from five interleaved runs.

The CUDA Graph geometric-mean improvement across the tested shapes is 16.57%, with no regression in any tested shape. The persistent Triton path provides the largest gain through 16 tokens; the general path remains 4.70%-5.43% faster above that range. CuTeDSL remains available for explicit tuning rather than default dispatch.

Test Plan

• pre-commit run --all-files
• Hyperconnection GPU numerics, dispatch, CUDA Graph, fixed-shape weight reload, PDL, and concurrent-stream tests: 37 passed
• CuTeDSL low-latency BF16 router tests: 29 passed
• Layernorm tests: 41 passed
• Kernel registry and selection tests: 237 passed, 108 platform skips
• Runtime kernel-boundary and weight-loader tests: 5 passed
• ServerArgs attention-backend tests: 22 passed

The focused Qwen4 test module is not collectable in the current local environment because its installed tuning dependency does not yet accept the tensor_initializers argument; the changed paths are covered by the focused runtime and kernel suites above.